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About Promising Practices in Online Learning 

Online learning within K-12 education is increasing access and equity by making high quality 
courses and highly qualified teachers available to students. Online learning programs offer courses, 
academic credits and support toward a diploma. They vary in structure, and may be managed by a 
state, district, university, charter school, not-for-profit, for-profit, or other institution. Thirty states 
and more than half of the school districts in the United States offer online courses and services, 
and online learning is growing rapidly, at 30% annually. This growth is meeting demand among 
students, as more than 40% of high school and middle school students have expressed interest in 
taking an online course. 

The most well established K-12 online learning programs are more than ten years old, and many 
programs have between five and ten years of operating experience. The newest programs are 
building on the expertise of those early adopters, as well as the experience of online learning in 
postsecondary institutions and the corporate world. A body of knowledge, skills and practices 
has been developed by individual programs, in collaboration with practitioners, researchers, and 
policymakers. Because there are so many types of online programs (full-time, supplemental, state- 
led, district-level, consortium), there are also many different approaches to teaching, student 
support, professional development, and other issues. 

This series, Promising Practices in Online Learning, explores some of the approaches being taken 
by practitioners and policymakers in response to key issues in online learning in six papers being 
released throughout 2008 and 2009: 

Blended Learning: The Convergence of Online and Face-To-Face Education 

Using Online Learning for Credit Recovery and At-Risk Students 

Management and Operations of Online Programs: Ensuring Quality and Accountability 

Socialization in Online Programs 

Funding and Legislation for Online Education 

A Parents' Guide to Choosing the Right Online Program 



The title, Promising Practices, deliberately avoids the term "best practices." There are too many 
approaches to online learning, and too many innovative teaching and learning strategies in the 
21 st century, for one method to be labeled "best." Instead, this series aims to discuss the issues 
and explore examples from some of the many online programs across the country, with a goal of 
illuminating some of the methods showing the most promise. 

Online learning offers the advantage of personalization, allowing individualized attention and 
support when students need it most. It provides the very best educational opportunities to all 
students, regardless of their zip code, with highly qualified teachers delivering instruction using the 
Internet and a vast array of digital resources and content. Through this series of white papers, we 
are pleased to share the promising practices in K-12 online learning that are already under way. 



Management and Operations of Online 
Programs: Ensuring Quality and Accountability 



Online learning is growing rapidly as states and districts are creating new online schools, and existing 
programs are adding new courses and students. The growth reflects the spreading understanding 
that online courses and programs can serve a wide variety of students and needs. These include: 

■ Creating opportunities for small and rural school districts to offer varied course subjects and 
highly qualified teachers to their students 

■ Allowing students to blend high school and post-secondary learning options 

■ Reducing class size 

■ Helping students recover credits in an alternative learning environment 

■ Providing individualized instruction and unique learning options 

■ Allowing students the opportunity to interact with students far beyond their school or town 
boundaries 

■ Meeting the needs and expectations of today's millennial students 1 

Many school leaders are excited about the possibilities of online learning. When they start an online 
school, however, they quickly confront all the challenges of managing a high-quality, successful 
online program: creating online courses; finding, hiring, and managing teachers; supporting 
students; managing technology; and evaluating their programs to determine if they are successful. 
Fortunately, many online schools have years of operating experience, have developed and revised 
formal operations and management structures, and provide examples of successful management. 
This paper explores emerging practices in online program management and operations that can be 
used by many people working with an online learning program, from executive-level school leaders 
to department managers to teachers trying to find ways to improve their effectiveness with online 
students. Although it does not address state or district policy issues, legislators and policymakers 
will find it useful to understand the varied approaches that online schools are embracing to ensure 
quality as they determine the best ways to create oversight while allowing innovation to meet the 
needs of students and schools. 



1 A longer list of the opportunities that online learning can provide, from which the above list is taken, has been developed by 
consultant David Glick and is provided in Appendix A. 
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The first step: determining online program type and goals 

The practices that online programs put into place vary according to their program type and 
goals. Two of the basic parameters are whether the program is full-time or supplemental, and its 
geographic reach. The full-time/supplemental parameter distinguishes whether the online program 
is responsible for students' state assessment scores, Adequate Yearly Progress, and other measures 
that are common to all public schools. The geographic reach determines whether the program 
has to operate fully at a distance or whether it can easily have a face-to-face component, and also 
determines the jurisdictional issues that a program faces (for example a multi-state program has to 
address content standards and teacher licensure across numerous states). Many management issues 
are dependent on one or both of these two parameters. 

In addition to the top-level program characteristics listed above, there are many other attributes 
that each program must determine for itself as it considers how best to manage its operations, 
including the types of students, whether teachers will be full-time or part-time, and whether 
curriculum will be licensed, home-grown or both. Determining these factors is a key component for 
a new program that is in its early planning stages, but understanding them can be useful as well 
for existing programs that are undergoing strategic planning and may ask themselves, "What are 
our characteristics now, and what do we want them to be in the future?" Figure 1 provides a list of 
attributes for consideration. 

Many of these elements operate along a continuum instead of being an either/or proposition. In 
addition, as programs grow and evolve over time, scalability is one of the major challenges that 
they face. Growth impacts every aspect of the organization, placing even greater importance on 
oversight and management. 

With the above characteristics in mind, the following sections review promising practices in the 
management and operations of online learning programs in the following areas: 

■ Curriculum development and course quality; 

■ Teacher management; 

■ Student support; 

■ Technology; and 

■ Program evaluation. 

For each topic, key issues and variables are identified, and examples of successful management 
are explored. 
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FIGURE 1: ATTRIBUTES OF ONLINE PROGRAMS 
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Curriculum development and course quality 



Issue 


Variables 


How can your 
program develop 
online courses and 
content that are 
of high quality, 
improve educational 
outcomes, and meet 
state standards? 


How many courses does your program have now? 


How many courses are you adding each year and how are those decisions made? 


Do you mostly purchase courses, develop your own, or mix and match? What 
information informs build or purchase decisions? 


What is your course review process and schedule (i.e., how often do you evaluate 
existing courses and plan new ones)? 


What is your course development and review budget and staffing? 



A key building block in overall program quality is the quality of the individual online courses. Online 
programs are evolving in their practices in this area, as the development and maintenance of online 
courses, which was once the domain of individual teachers, has evolved in many schools that now 
use teams of teachers, content experts and instructional designers to ensure high-quality courses. 

As the technology behind online courses and learning management systems becomes more robust, 
the need for specialized skills becomes more acute. The potential for interaction and the use of 
multimedia, simulations, and gaming to increase student engagement increases the range of 
knowledge and expertise necessary to create online content. 

The makeup of course development teams varies, but most teams consist of a project manager, 
course/content writers, instructional and information designers, multimedia developers, and 
copyeditors and proofreaders. When looking at an online school's course development process, 

"A good rule of thumb is to ask 'What are the qualifications of the curriculum team?'" says Bror 
Saxberg, Chief Learning Officer at K 12 , a provider of curriculum, technology and school services for 
online programs, and the largest operator of full-time online schools in the country. "Look for a 
mix of teachers, instructional designers, information designers, a solid base of research of learning 
theory, multimedia experts and technologists." 

A consistent theme among online course developers is the huge difference between an online class 
and a face-to-face class, in both teaching and course development. "There is a significant difference 
in writing prose [narrative] and writing with an online voice," notes Jonathan Schmalzbach, Director 
of Content Development at Apex Learning. "Crafting a script that complements a multimedia 
presentation of a concept is a new skill for most teachers and a difficult one to find. It's a real coup 
and pleasure to find a great classroom teacher that can translate that experience to online content 
presentation, someone who understands how visuals, audio, text and other content elements 
combine in a storyboarding process." 

Many online schools recognize that the act of teaching a class is quite different from the process 
by which an online course is created. In a physical classroom, a teacher may divide the learning 
experience into categories that include the textbook, the class lectures and activities, and additional 
instructional materials. The online environment mixes and matches these components. A physical 
textbook may no longer be used, increasing the importance of the way in which educational 
materials and the teacher's instruction work together. While online teachers benefit from 
professional development in teaching online, creating an online course requires an additional set 
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of skills. The Virtual High School Global Consortium (VHS), which offers extensive professional 
development options to both online and classroom teachers, separates teacher professional 
development in its member schools into two parts: Netcourse Instructional Methodologies for 
those who are teaching an existing VHS course, and Teachers Learning Conference, which prepares 
face-to-face classroom teachers to become online course developers and instructors with VHS. 

KC Distance Learning takes a similar approach, with a professional development course for course 
writers that includes how to incorporate student learning skills, how to develop higher order 
thinking skills, and how to create a course to align with state standards. 

The fact that courses are now commonly built by a team that often no longer includes the course's 
teacher creates a new set of challenges. "The course development team can build a wonderful 
exercise, with excellent rubrics and teaching notes, but if the instructional department does not 
respond with adequate training, the implementation of the exercise will likely go wrong and 
instructional value can easily be lost," notes education consultant John Adsit. "It may be as simple 
as telling instructors not to judge student performance in threaded discussion on the number of 
entries, but rather to focus on the contribution of meaningful substance, or requiring students to 
define individual roles in a group exercise so that each student's work and participation can be 
evaluated separately." 

Build versus buy 

Online curriculum may be developed internally by the online program, may be licensed from an 
outside source, or may consist of some combination of both. Data reported by Keeping Pace with 
K-12 Online Learning in 2007 based on a survey of 60 online schools suggest that the percentage 
of courses that are licensed or built in-house is highly variable among online programs. Programs 
license anywhere from none to all of their courses from outside providers. Indeed, the symmetry 
among the percentages of courses being licensed versus being developed in-house was remarkable: 
23% of online schools had licensed all of their courses, and 23% of programs had developed all of 
their courses; 53% had licensed half or more of their courses, while 55% had licensed half or less of 
their courses. 

Reasons for using curriculum licensed from an outside source include: 

■ The expertise of vendor development teams, including writers, instructional designers, 
multimedia developers, and technologists, often exceeds the expertise within the online 
program. 

■ A wide variety of curriculum and specialty courses is difficult to produce in-house. 

■ A program may lack staffing, funding, and/or expertise to develop and update high 
production-value content. 

■ Organizations that are focused on curriculum development often have the resources to 
incorporate more extensive user testing and feedback than individual schools. They also may 
be more equipped to provide regular updates, including maintenance of multimedia-based 
content over generations of versions and changing technology. 

Outside content ranges from individual learning objects that are incorporated into existing courses, 
to full courses that are not meant to be edited, with a range of options in between. In either case, 
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licensing content from an outside provider does not free the online school from the responsibility of 
evaluating the course content, and integrating it with successful teaching strategies. 

Reasons to develop content internally include: 

■ The need to adhere to state and district standards and greater confidence that home grown 
courses will do so. 

■ Linking content creation to teaching online in a way that involves teachers at a greater level 
than licensed content may allow. 

■ The expense of licensed curriculum, especially compared to using teachers or other staff who 
are full-time. 

■ Support for specific instructional philosophies not supported by course vendors, such as 
project- or inquiry-based learning. 

Course standards, quality, and revisions 

Online courses are typically designed to meet national and/or state content standards. Courses and 
course content licensed from outside sources are usually modified as necessary to meet state and 
possibly district standards. Advanced Placement courses must meet the requirements of the College 
Board. However, these standards are only a beginning. 

A process of evaluation, feedback (from both students and teachers), and revision leads to 
accountability and improved quality. Initial development of courses is only a start, as they should 
be revised regularly to keep them current in terms of both content and technology, which includes 
everything from including the latest events in a history course to making sure that links are still live 
(and going to the intended information). Course revisions can take two approaches, continuous 
modification or revisions on a set schedule — or a combination of both. Some programs make small 
ongoing changes based on student and teacher evaluations, course failure rates, service ticket data 
from support desks, or changes in external standards, and use regularly scheduled revisions to 
ensure that larger changes to instructional design and multimedia content are done. 

Feedback from students and teachers helps schools determine what changes need to be made to 
existing courses, and provides ideas for new courses. "We create a student survey each spring and 
ask students what courses they want to see created," says Tracy Quarnstrom, Director, TRIO Wolf 
Creek Distance Learning Charter School. "We also use a survey at the end of each existing course to 
see what needs to be improved. With changing standards in Minnesota some classes are no longer 
needed as they do not align to the [state] standards, while other topics are in growing demand." 

iNACOL's National Standards for Online Course Quality addresses many of these issues, and others 
as well. The comprehensive standards allow online programs to assure quality internally, and 
demonstrate quality to outside stakeholders. 
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iNACOL's National Standards for Quality of Online Courses 

iNACOL's standards provide guidance in several key areas of course development and delivery. The 
following list is representative of the standards, but is not comprehensive . 2 

Course content 

■ The course goals and objectives are measurable and clearly state what the participants will know or 
be able to do at the end of the course. 

■ The course content and assignments are aligned with the state's content standards or nationally 
accepted content standards set for Advanced Placement courses, technology, computer science, or 
other courses whose content is not included in state standards. 

■ Issues associated with the use of copyrighted materials are addressed. 

Instructional design 

■ Course design reflects a clear understanding of student needs, and incorporates varied ways to learn 
and multiple levels of mastery of the curriculum. 

■ The course unit overview describes the objectives, activities and resources that frame the unit. It 
includes a description of the activities and assignments that are central to the unit. 

■ Each lesson includes a lesson overview, content and activities, assignments, and assessments to 
provide multiple learning opportunities for students to master the content. 

Technology 

■ The course architecture permits the online teacher to add content, activities and assessments to 
extend learning opportunities. 

■ The course accommodates multiple school calendars. 

■ Hardware, web browser and software requirements are specified. 

Student assessment 

■ Student evaluation strategies are consistent with course goals and objectives. 

■ The course structure includes adequate and appropriate methods and procedures to assess students' 

mastery of content. 

■ Assessment strategies and tools make the student continuously aware of his/her progress in class and 
mastery of the content beyond letter grades. 

■ Assessment materials provide the flexibility to assess students in a variety of ways. 

Course evaluation and management 

■ The results of peer review and student evaluations of courses are available. 

■ The course is evaluated regularly for effectiveness, and the findings are used as a basis for 

improvement. 

■ The course is updated periodically to ensure timeliness. 

21 st century skills 

■ The course intentionally emphasizes 21 st century skills in the course, including using 21 st century skills 
in the core subjects, 21 st century content, learning and thinking skills, self-directed learning, global 
awareness, and includes 21 st century assessments. 



2 For the full list of standards please see http://www.iNACOL.org/resources/nationalstandards/index.php 
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Teacher management 



Issue 


Variables 


How can your 
program ensure 
that it has the best 
teachers, with the 
skills and experience 
necessary to 
successfully teach 
online? 


How many teachers does your program have, and are they primarily full-time or 
part-time? 


How does your program recruit new teachers? 


Are your teacher orientation and professional development programs meeting the 
needs of your faculty and goals of the program? 


What mentoring, support, and accountability processes are used in your teacher 
management operations? 


Does your program have teacher-student communication requirements and how is 
the effectiveness of the requirements measured? 



Teaching is just as important in the online classroom as it is in the physical classroom. Successful 
online programs use many of the following practices to ensure the highest quality faculty for their 
students. 



Recruiting 

Because few pre-service teachers receive much training in online instruction in colleges of education, 
the recruiting and initial training of highly qualified teachers to make the move to online teaching 
is a critical component of success. Most programs have a clear strategy for online teacher recruiting 
and training, but the criteria and techniques used vary significantly from program to program. 

Most programs agree that new online teachers must 1) put their role as a facilitator of student 
learning above other aspects of teaching, 2) have the ability to adapt and manage change, 3) have 
a high level of content mastery, and 4) be ready to make the shift to online instruction. However, 
the philosophies and techniques used to evaluate prospective online teachers vary from program 
to program. Several online schools and companies, including Illinois Virtual High School, conduct 
the entire recruiting process electronically, narrowing the field of candidates to those comfortable 
with the use of online technology and able to demonstrate a capacity for online communication. 
Other schools feel that the technology aspects are fairly easily taught, so they don't feel the need 
to vet the teachers' technology skills as an initial step. "I'm looking for teachers with superior 
teaching and communication skills, those that love to help students learn," says Jack Babani, 

Director of Instruction at Apex Learning. "We feel most any qualified teacher can be trained on the 
technology components, but not content knowledge and the ability to relate to students." Apex's 
recruiting is done through conversation and face-to-face meetings, putting the teacher behind the 
student desk using role-playing and other interaction to the point where teachers often self-screen. 
Some programs start the recruiting process with commercially available tests designed to gauge a 
teacher's strength in focusing on the student, rather than on curriculum or policy. Florida Virtual 
School (FLVS), for example, first identifies teachers that are student-centric. Jeff Murphy, one of the 
Directors of Instruction at FLVS, notes "Teachers moving to online learning must be comfortable 
with the amount of change. The technology, tools and instructional methods change constantly, so 
we are always probing to make sure our teachers embrace change." 

Most educators tasked with hiring teachers agree that training begins with the first recruiting 
phone call or interview between the prospective teacher and the program. Several programs 
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stress the importance of making teachers aware of the challenges of online teaching in this first 
interaction. "Teachers should have no illusions of the difficulty of teaching online," notes Bror 
Saxberg, Chief Learning Officer at K 12 . "Most think it is as tough as the first year of teaching in 
a physical classroom." Interview techniques that help recruiters avoid potential teachers who are 
more interested in the lifestyle of working from home than on tackling a new teaching challenge 
are important because the desire to work from home, or the misguided belief that online teaching 
requires less time than teaching in a physical school, are poor reasons to move to online teaching. 

Scalability is one of the major challenges faced by rapidly growing online learning programs. K 12 
is facing unprecedented growth in the teaching staff at schools across the country, growing from 
1,200 virtual teachers last year to 1,600 for the 2008-2009 school year. Teresa Scavulli, Senior 
Director, Teacher Effectiveness at K 12 , believes "One of our most pressing issues is scalability. We 
have to be able to move quickly to deliver educational services, almost on an as-needed basis." 

Professional development 

Professional development for online teachers begins with orientation and extends throughout 
the teacher's online career. Establishing a standard training program at the point of teacher 
induction gives the instructional staff a common starting point and a baseline from which to 
evaluate performance. For example, the Virtual High School Global Consortium requires all new 
online teachers within its nationwide network of schools to complete the Netcourse Instructional 
Methodology course before they are certified to teach. This intensive 10-week graduate course 
covers the skills necessary to ensure a successful start to online teaching, including the pedagogy of 
online teaching, course management system basics, and online classroom management techniques 
that the teacher needs to be successful. 

Continued growth through professional development is critical, in particular because online learning 
technology and pedagogy is changing quickly. Topics that are often part of continued professional 
development include: 3 

■ Helping teachers understand how to motivate individual learners 

■ Enhancing student interaction and understanding without visual cues 

■ Tailoring instruction to particular learning styles 

■ Developing Web 2.0 and 21 st century skills 

■ Modifying interactive lessons to meet individual student needs, including the needs of 
disabled students 

■ Developing heightened communication skills to enhance email correspondence and discussion 
board postings, and to recognize the tone of writing and the nuances of word usage 

■ Improving the time management skills critical for online teachers 

Many online programs are now implementing extended professional development opportunities 
that grow as teachers' skills and techniques grow. For example, the VHS Global Consortium has 
introduced a new five-course series, 21 st Century Teaching Best Practices, that is designed for all 



3 This list is from A National Primer on K-12 Online Learning, published by iNACOL, and was based on Essential Principles of Online 
Teaching: Guidelines for Evaluating K-12 Online Teachers, Southern Regional Education Board, April 2003 
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levels — from classroom teachers who want to add an online component to their face-to-face class, 
to experienced online teachers looking for the latest pedagogy and online methodology. These 
courses explore the technology literacy and Information and Communication Technology (ICT) skills 
needed for effective teaching and learning, including how face-to-face teachers can incorporate 
online resources and tools into their classroom instruction, including Web 2.0 tools, WebQuests, 
blogging, wikis and open educational resources (OER). 

K 12 professional development trainers are all top-performing, master teachers with extensive online 
teaching experience. This level of expertise brings credibility with new teachers and allows them to 
work as mentors and role models. Professional development is organized into four national regions, 
providing greater opportunities for face-to-face training that supplements national training efforts. 

K 12 offers weekly online sessions for teachers grades K-8, and once a month sessions for 9-12 
teachers; topics include state testing, intervention for one-to-one instruction, peer counseling, and 
introducing teachers to a new student-parent community website. 

Some online programs are tapping into partnerships with outside educational institutions to enhance 
their internal professional development. Connections Academy, for example, has partnered with 
Boise State University to tap into the university's expertise in teacher training (along with curriculum 
development and evaluation). As an increasing number of universities offer programs for existing 
and pre-service online teachers, the opportunities for such partnerships will improve. 

Teaching practices and evaluation 

Successful online learning often is based on consistent communication between students and 
teachers, and clear feedback from teachers to students. To ensure that such communication 
and feedback is occurring, many online programs — and some state policies — have established 
communication requirements. Examples of such guidelines include: 

■ Student email is answered within 24 hours 

■ The teacher initiates phone calls to each student at least monthly 

■ A weekly progress check is recorded for each student 

■ Student discussions are actively facilitated 

■ Parent calls are held monthly 

■ Weekly or monthly face-to-face meetings are held with a mentor or supervisor 

Clear guidelines and expectations ease the evaluation process, helping teachers know what is 
expected of them. Online teaching practice, however, is still in its early stages, with limited research 
to help teachers and supervisors understand what works best. The lack of commonly accepted best 
teaching practices combined with the distance at which many teachers work creates a challenge for 
managers. "We don't yet have a good sense of what is required work time and what is preparation 
time that all teachers put in as an addition to their typical work day," notes Tracy Quarnstrom, 
Director, TRIO Wolf Creek Distance Learning Charter School. Other practitioners note that many 
online teaching practices are still just being established and that there is little definitive research on 
the specific approaches that work best for students. 
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Although research into effective online teaching is not yet extensive, iNACOL has published a set 
of online teaching standards that provide an excellent starting point. In addition to professional 
development and communications standards similar to those listed above, other standards include 
reguiring that the teacher: 

■ Plans, designs and incorporates strategies to encourage active learning, interaction, 
participation and collaboration in the online environment 

■ Provides online leadership in a manner that promotes student success through regular 
feedback, prompt response and clear expectations 

■ Models, guides and encourages legal, ethical, safe and healthy behavior related to 
technology use 

■ Understands and is responsive to students with special needs in the online classroom 

■ Demonstrates competencies in creating and implementing assessments in online learning 
environments in ways that assure validity and reliability of instruments and procedures 

■ Develops and delivers assessments, projects, and assignments that meet standards-based 
learning goals and assesses learning progress by measuring student achievement of learning 
goals 

■ Demonstrates competencies in using data and findings from assessments and other data 
sources to modify instructional methods and content and to guide student learning 

■ Demonstrates freguent and effective strategies that enable both teacher and students to 
complete self- and pre-assessments 

Mentoring and monitoring online teachers 

Successful online programs often prioritize a culture of teacher collaboration and mentoring. 
Although establishing clear and consistent procedures is critical to establishing accountability, 
creating a sense of collaboration and mentoring fosters a collegial work environment that 
encourages teachers to contribute innovative ideas. For example: 

■ FLVS uses a co-teaching approach that mimics some of the techniques being successfully 
employed in brick and mortar institutions to provide better one-to-one instruction and 
teacher availability, and a team-teaching approach largely defined by the FLVS teachers 
themselves. Each teacher team decides how they prefer implementing shared instructional 
tasks, from monthly student phone calls to being on call for student questions, or sharing 
the tasks of lesson planning and grading duties. All teachers are supported by a team of 
educators including trainers, content buddies, mentors, instructional coaches, instructional 
managers and instructional leaders. 

■ K 12 has established a Founder's Club to recognize outstanding teaching efforts within its 
community of teachers, and to gather and use feedback from this group to identify and 
address teacher issues from across the country. Teachers are recognized based on a set of 
quantitative and qualitative standards. Selected Founder's Club teachers attend an annual 
conference and work year-round to help with student retention, socialization and community 
building, student achievement, parent training and student motivational issues. Feedback 
from this group and other teachers led K 12 to launch a study of high performing teachers. 
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New teachers also need to understand, as most program directors already know, that they must 
be able to demonstrate accountability on a class-by-class and student-by-student basis. In the 
traditional brick and mortar institution, once the classroom door is closed there is very little oversight 
of the instruction and learning taking place in the room. Online learning is far more transparent and 
is accessible by design, making it easy to monitor most aspects of student and course progress on a 
daily basis. 

Academic integrity 

Online teachers and managers must also devote some of their time and practice to ensuring the 
academic integrity of their classes and program. Questions and concerns regarding academic 
integrity have been raised since the early days of distance learning and have continued with the 
growth and acceptance of online learning. Most questions about the authenticity of student work 
come from those outside the community of online learners, while most teachers and administrators 
feel the issue is addressed by a number of established and proven assessment methods. Violations of 
student academic integrity are not unique to the online environment, but it is still perceived to be a 
challenge and a continuing important issue in online education. 

Academic integrity has been defined as a commitment, even in the face of adversity, to five 
fundamental values: honesty, trust, fairness, respect, and responsibility . 4 Some of the threats to 
academic integrity include ignorance, pressure, stress, and time management issues. 

Online programs address academic integrity issues in several ways: having staff members dedicated 
to the issue, ensuring consistent communication between teachers and students, requiring at least 
some assessments to be proctored, and using software to check for plagiarism. 

Florida Virtual School, for example, has used all of these approaches. A full-time Academic Integrity 
Specialist is employed to establish and revise academic integrity policy and programs, and to work 
with teachers via monthly synchronous training sessions and one-on-one when necessary. Teachers 
have phone calls with students at least monthly, engaging students in a one-on-one conversation 
about the content, asking probing questions that generate topical discussions. If there is an 
incongruity between the student's daily work online and the level of comprehension demonstrated 
during the phone instruction, the teacher can identify potential abuses and work to correct them. 
Finally, random proctored testing provides an institutionalized approach to ensure student integrity 
while avoiding any perception of persecution when proctored testing is requested by a teacher. 



4 The Fundamentals of Academic Integrity. The Center for Academic Integrity, Duke University. October 1999. 
http://www.academicintegrity.org/fundamental.asp 
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Student support 



Issue 


Variables 


How can your 
program ensure 
that your students 
have the orientation, 
technical support 
and academic 
counseling they 
need to be 
successful? 


Is your program full-time, supplemental, or both? 


Do most students access courses from school, home, or some other location? 


Is your student orientation process sufficient and well-executed? 


How are student progress and activity tracked, and who has access to that 
information? 


Do your students receive robust technical support regardless of how they access 
their online courses? 


How are you supporting students with physical or learning disabilities? 



Student support is an integral part of online school success. Support includes technical help (i.e., 
accessing the course, computer hardware and software, communication tools), academic issues 
(course content and instruction, tutoring, counseling) and administrative support (enrollment, 
personal information). Successful online programs have adopted a variety of practices to ensure top- 
quality student support, including the practices described below. 

Student support structures 

The ways in which student support is provided varies greatly depending on whether the online 
program is supplemental — meaning the student has access to support in a local physical school — or 
full-time — in which case the student may be helped by a parent or other learning coach in addition 
to virtual support provided by the online program. 

Many supplemental online programs require that physical schools provide on-site coordinators 
who assist students, while also helping adapt the school's procedures to accommodate online 
learning. IDEAL-NM, one of the newer state-led online programs, imported the philosophy and 
strategy of on-site coordinators at local school partner sites from other successful online learning 
programs. Site coordinators provide on-site support for students, teachers and parents, explaining 
the advantages of IDEAL-NM to students, helping with student registration, monitoring academic 
progress, intervening if any problems occur and in some cases working with students to provide 
instruction in computer labs. "The first step in establishing a partnership with a school is to identify 
the site coordinator," says Tim Snyder, Executive Director of IDEAL-NM. "A good site coordinator is 
like gold to our program. They become our knowledgeable and passionate ambassadors and set the 
tone for how well the school implements the online program." Dawn Nordine, Executive Director of 
Wisconsin Virtual School — the Wisconsin Web Academy — agrees. "The support of the local mentor, 
or Local Education Guide, is critical to the success of most students taking supplemental courses 
online. We have increased our opportunities for face-to-face training for these folks who support 
the students daily on a range of activities: placing the student, choosing the appropriate courses, 
registration, teacher communication, and policy development." The Idaho Digital Learning Academy 
(IDLA) has taken the need for strong local support a step further, by reducing the course fees for 
districts that have had a site coordinator go through an IDLA professional development course. 
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Enrollment and orientation 

The first contact between a prospective online student and the online school is an important step in 
ensuring a successful transition to online learning. Established procedures help guide online learning 
staff in how to mentor the student through the process of enrolling and participating in online 
courses, covering issues such as: 

■ An initial checklist of points to cover with students and parents during the initial period 

■ A pre-enrollment survey to challenge students' preconceptions of online learning and to 
determine their level of readiness for this new modality 

■ A student orientation course before the first academic course to set performance 
expectations, familiarize the student with the learning management system and identify any 
technical support issues 

■ A review of online learning policies to cover grading requirements, student discipline, and 
warning and probation policies 

Technical support 

Student access to robust technical support relieves one of the key barriers to student success in 
online learning while taking a significant burden off teachers. To reduce initial technical support 
calls, many programs use automated checks of bandwidth, versions of Flash, Acrobat, and Java, and 
other plug-ins required of students. Tutorials and online orientation sessions familiarize students with 
the learning management system to assist in a smoother transition to the online environment. Many 
programs make help desk support available 24/7 both by phone and virtually, and many have or are 
instituting service ticket systems to track more carefully student support communication and results. 

In some cases, students are taking support into their own hands. SHOCK (Students Helping Others 
Collect Knowledge) is an innovative new student support program at Florida Virtual School that was 
envisioned and executed by past students to provide peer-to-peer support to help students with 
everything from how to navigate and use the learning management system, to how to study for a 
math exam. 

Academic support 

Technical support is just one component of helping students learn online; the other key component 
is academic support. This may include, but is certainly not limited to, the following set of tools and 
techniques to ensure a smooth transition to online learning. In some programs, these are handled by 
teachers, while others have features for such support built into their learning management systems. 
In other programs, these are functions performed by either face-to-face or virtual non-teaching staff. 

■ Assessments of student progress at regular intervals 

■ Support materials like student handbooks 

■ Guidelines for conduct (both students and teachers) 

■ Organization of students into groups/cohorts 

■ Tutoring services 

■ Pre-course tests that measure a student's readiness for the online learning experience 
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■ Required amount of course access 
time and how that differs from 
time on task 

■ Expectation of the parent and 
family roles in the student's 
experience; parents' role in 
monitoring course progress, 
conference calls with teachers, 
face-to-face opportunities for 
students and parents 

Communication is the key element of 
student support. Students should never 
feel that they don't know where to 
turn with a question or problem, and 
should be able to easily reach their online 
teacher, school facilitator, learning coach, 
or other source of assistance. 

Counseling and mentoring 

Physical schools provide counseling and 
mentoring to students. Online schools are 
moving in this direction as well, either by 
instituting online counseling support for 
students, or extending the training for 
on-site coordinators who often perform 
many of the tasks of a traditional school 
counselor. 

School-based counseling comes with a 
heavy load of administrative duties (credit 
analysis, senior credits for graduation, 
community service hours) that limit 
personal contact with students and 
provide little opportunity to be proactive. 

In supplemental online learning situations, 
the online program may be supporting 
the district's counseling efforts, so the 
online counselor is freed to focus on 

students: establishing regular communications, leading students to online resources published by 
the online program, answering general questions, or encouraging the student to look for resources 
outside the school system when necessary. The site facilitator in a supplemental program may or 
may not have roles defined in tandem with the physical school counselor — in some cases the two 
roles may be taken on by the same person. 



Working with local school districts 

Most schools are excited by the possibilities 
of online learning, but concerns arise once the 
reality of implementing online instruction sets 
in; how do we make this happen? The Hamilton 
County Virtual School (TN) and IDEAL-NM 
distribute a handbook to districts ready to have 
their students take supplemental online courses. 
The handbook gives school administrators and 
on-site coordinators the procedures, application 
documents and policies necessary to get 
the program off the ground. The IDEAL-NM 
handbook, for example, outlines a step-by-step 
process including: 

■ Defining the roles and responsibilities of 
on-site people with "getting started tips" 
for each position 

■ Explaining the costs and funding of the 
program 

■ Describing the learning management 
system and other technology 

■ Providing program policies, from 
acceptable use and attendance policies, 
to how the program and partners should 
handle conduct and discipline problems 
and withdrawals. 

This handbook, and the procedures and 
policies it outlines, eases concerns for school 
administrators, demonstrates the level of 
support IDEAL-NM provides, and makes 
program implementation easier by providing a 
blueprint to success. 
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Some supplemental online programs, and most full-time online schools, have full-time online 
counselors. At FLVS, the counselor's role begins with student recruitment and orientation and 
includes responsibilities ranging from monitoring student progress to collaborating with online 
teachers to mediate performance and behavioral issues. Students are often more expressive with 
their online counselor, comforted by the relative anonymity of email and phone communication as 
compared to face-to-face meetings that can be intimidating for students. "The online environment 
allows for more one-on-one connection than I had as a counselor in a physical school," according 
to Patty Cordones, Online Counselor for Florida Virtual School. "Student needs range from those 
as casual as a complaint about a course being too rigorous to something as profound as a suicide 
intervention. As counselors, we're trained to follow defined procedures when serious problems arise; 
call the parent, the teacher and district, and find help for that student." 



Technology management 



Issue 


Variables/Elements 


How can your program 
develop a technology 
strategy and systems to 
improve student and teacher 
learning, and provide data 
for the daily management 
and oversight of staff, while 
anticipating change and the 
need for innovation? 


Does your program have a clear set of functional criteria and operational 
objectives for the learning management system (LMS) application? 


Does the LMS integrate with the program's student information system (SIS), 
and is the right student data being captured and used effectively? 


Does your program have an annual evaluation of its technology needs and 
performance to keep pace with the rapid rate of technological change? 


Can existing technology be scaled to accommodate program growth and 
adapted to new instructional technology tools? 



While online educators often point out that teachers, and not computers, are at the heart of online 
learning, technology systems are clearly an important component of an online school. Computer 
hardware and software are essentially the facilities of an online school, much as classrooms and 
buildings are the facilities of a physical school. These tools not only provide information and data to 
manage the program, but also help teachers become more innovative and effective at their jobs. 

Following are some of the practices successful online programs employ to ensure that their 
technology systems contribute to overall program quality. 

The role of LMS and SIS technology 

Online education is first and foremost based on the interaction between the student and the 
teacher, but technology plays a significant supporting role in delivering instruction and an integral 
role in providing accountability and management tools. The technology at the heart of an online 
school is the learning management system (LMS) — the set of tools that houses course content and 
provides the framework for communication between students, teachers, and parents. The student 
information system (SIS), which manages student data, is the other main software component. 
Teaching online through an LMS presents several advantages compared to a physical classroom, 
including: 
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■ Monitoring student performance, including observing individual student trends quickly using 
real-time online data; also tracking of attendance to meet state requirements for funding; 
and to meet regulatory demands 

■ Organizing student learning by age groups; younger students may have difficulty sitting at 
a computer for hours, while older students can make better use of video, animations, and 
simulations 

■ Allowing for quick updating of curriculum 

■ Scheduling and tracking for families of full-time students, allowing easy monitoring of 
assignments and tasks completed 

■ Adapting instruction to adjust to the student based on the difficulties being encountered 

Keeping up with improvements in technology infrastructure 

Experienced online programs commonly make a point to "know your core competency and 
outsource the rest," noting that they should not get too caught up in technological change and 
lose their focus on education. "VHS Global Consortium stays focused on our core competency 
and mission, to educate students," says Liz Pape, CEO. "We are not a data support center or an 
LMS developer, but it's critical that we maintain strategic management staff to closely monitor 
our vendors and ensure the technology we employ from outside the organization provides the 
information and data we need to manage the program and manage the entire organization." 

Programs must evaluate new technology and recognize when the tools are largely "bells and 
whistles" and when they truly advance instructional practice and value. It is far too easy to get 
caught up in the possibilities of an instructional technology before it is reviewed and tested by those 
with the daily experience to judge its merits. 

Equity and privacy issues impact technology oversight decisions and processes. Technology 
departments must establish policies to protect student data and information, as well as meet 
standards for access by all students regardless of connectivity or disabilities. 

Regardless of the learning management system and administrative software in place, programs 
must plan for future flexibility and growth, not just daily operations of the existing systems. "Be 
prepared for change in the world of technology, meet regularly as a team and build in professional 
development and training for the entire technology staff," says Jonathan Beckham, FLVS Senior 
Manager Information Services. 
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Program evaluation 



Issue 


Variables 


Is your program 
meeting its mission 
and goals as well as 
the expectations of 
the stakeholders and 
the community? 


What is your program's mission; does the mission include specific student 
populations? 


Are student outcomes meeting program and state expectations? Can improved 
student outcomes be demonstrated? 


Does your organization conduct an annual program evaluation? 


How satisfied are stakeholders, including students, schools, and parents, with their 
experience with the program? 



Because online learning is still relatively new, online schools sometimes have to demonstrate quality 
and results in ways that go beyond the requirements that physical schools meet. Supplemental 
programs, in particular, face the challenge of not being evaluated in the same way as public schools, 
because they are not responsible for their students' state assessment scores (in most cases). One 
way that online schools address quality and performance concerns is by commissioning regular 
program evaluations. 

Internal and external evaluations 

Program evaluations fall into two categories: internal (conducted by the program staff) or external 
(conducted by someone outside the organization). Internal evaluations have the advantage of 
timeliness, as they can generally be scheduled conveniently at any time during the year, and staff 
that have a thorough knowledge of the program being evaluated. 

External evaluations bring a fresh look at a program from someone removed from the pressure of 
stakeholders or program staff. There are no personal relationships to cloud the examination of the 
program, and an outside perspective may bring flaws to light that go unseen by staff too close to 
the situation to recognize the problems. Also, external evaluations often carry greater validity with 
stakeholders. However, the cost of external evaluations can be high and the process takes additional 
time to bring the evaluator up to speed on the processes and metrics of the program being 
evaluated. 5 

Pieces of the evaluation puzzle 

Although the specifics of an evaluation vary significantly, an evaluation usually starts with an 
examination of the program's mission to determine whether it is meeting its organizational goals. 

If the goal of an online program is to give students more opportunities for recovering course 
credits in order to graduate, then two evaluation measures could be the number of credit recovery 
courses being offered, and the graduation rate for students in those courses. If the mission of the 
online program is to increase 21 st century learning opportunities for a district's students, possible 
evaluation criteria include 1) the number of new courses available to students; 2) the increase in 
student proficiency in use of Web 2.0 tools; and 3) the increase in teachers' use of Web 2.0 tools in 
classroom and online instruction. 



5 Evaluation in Online Learning, L. Pape, M. Wicks, C. Brown, and W. P. Dickson, in Keeping Pace with K-12 Online Learning, 2008, 
www.kpk12.com; further examples are from this source as well 



20 



Management and Operations of Online Programs: Ensuring Quality and Accountability 





Evolving Models for Performance Data Systems: Florida Virtual School's 
Virtual School Administrator 



Florida Virtual School (FLVS) is not only the largest online program in the country, but it is 
also the only program whose funding is based primarily on student achievement instead of 
measures such as seat time, which are common across K-12 education. With funding based on 
student achievement, consistent tracking of course progress based on competency becomes 
critically important not just for students and teachers, but for administrators as well. The 
FLVS Virtual School Administrator (VSA), a custom program integrated with the program's 
course management system, has been instrumental in helping FLVS meet this challenge. 

VSA provides sophisticated functionality to measure ongoing competency throughout each 
course and performance levels for each student, and consolidates data from the LMS for use 
by teachers and administrators. The technology staff works closely with instructional and 
course development personnel to track data and create reports that meet the needs of each 
department. Dashboards have been created to provide at-a-glance insight into a student's 
course progress, grade status, a teacher's monthly phone call log, and responsiveness to 
email, among other data points. All of the operational departments of FLVS participate in 
defining the data and applications they need to improve their effectiveness and provide a 
competency-based learning experience for students. The FLVS technology group is working to 
add predictive analysis capability to the existing system, using historical data and patterns 
to predict and improve student performance. 

Although in-house software development is sometimes the best strategic path, FLVS wants 
to spend its time educating students — not responding to technology-related help tickets 
from students and teachers. While the development of VSA has been handled in-house to 
accommodate the needs of the stakeholders, FLVS made the strategic decision to outsource 
student help desk and first-line teacher technology support. With the large volume of help 
tickets encountered by FLVS, a software tool was selected that routes questions and problems 
to the appropriate technical staff for quick response and generates metrics on the instances 
of help tickets, the types of problems, how quickly the problems were resolved and who had 
to be involved in the solutions. Jonathan Beckham, Senior Manager Information Services 
at FLVS, stresses, "You are not done with the decision on which vendor to use and which 
software package to buy. With all outsourcing, it is critical for the organization to maintain 
technical staff to manage and monitor the work of the vendor. Use outsourcing when available 
and reliable, but be vigilant and establish internal oversight of and management of vendors." 



Evaluations are commonly based on one or more of stakeholder surveys, outcomes data, and 
reviews of internal processes. 

■ Surveys may be done of students, parents, teachers, educators in school districts using 
supplemental online courses, and other stakeholders. They may be done at multiple times 
of year; for example, students may be surveyed every semester while parents are contacted 
once per year. 
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■ Outcomes data vary based on program type. Supplemental programs often rely on course 
completion rates and results of Advanced Placement exams, while full-time programs can 
report data of state assessments and other measures common to all public schools in a state. 

■ Internal processes such as course development may be benchmarked against other 
programs, or standards such as those published by iNACOL. In addition, the evaluation may 
report financial information, staffing levels, and similar organizational metrics. 



Conclusion 

Online learning promises cost-effective solutions to the challenges education leaders face in many 
areas, particularly in difficult economic times. States and districts are increasingly turning to online 
programs to expand educational opportunity, equity, and access while individualizing learning 
options for students. This growth in online learning requires that school managers be able to 
effectively operate and assess their programs. 

While most online programs deliver quality curriculum, effective teaching, and improved student 
outcomes, such positive outcomes are not assured. They are instead a result of forethought and 
diligent management, starting with defining and planning the online program, and extending 
through all aspects of management and operations. 

The growth of online learning programs, the demands of expanding course offerings, and the 
need to service different types of students and deliver learning in a variety of formats are just a 
few elements that illustrate the complexity of running a quality online learning program. Although 
many existing online learning programs started as an individual vision, created and fostered by a 
small, dedicated team of teachers and administrators, most programs now recognize the need for 
processes and procedures that ensure program quality and accountability. 

Managing growth, and keeping up-to-date, are two of the issues that online programs commonly 
face. "One of the next challenges in program management is staying current: currency in teaching 
skills, currency in course design standards, and currency in technology strategies as broadband 
access and new online tools bring new instructional opportunities to bear," says Liz Pape, CEO, VHS 
Global Consortium. "Ubiquitous computing will be demanded by students and improve teaching 
and learning, but it will require constant evaluation and integration into existing programs." 

Online learning is already improving student outcomes, and holds the potential to be a truly 
transformative element of education in the years ahead. Clayton Christensen, Harvard Business 
School Professor and one of the authors of Disrupting Class: How Disruptive Innovation Will Change 
the Way the World Learns, predicts that in about a decade, half of all high school courses will be 
online. But previous trends do not guarantee future growth, and online educators recognize that 
their programs face far greater scrutiny than physical schools. 

Regulators and policymakers at the state and federal level often grapple to find the appropriate 
balance between mandating quality and accountability on the one hand, and allowing room for 
innovation on the other. Programs that can demonstrate a sensible approach to ensuring quality 
content in their courses, highly qualified and well-trained teachers, comprehensive student support 
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services and forward-looking technology systems can help guide constructive policy and regulatory 
oversight. 

"Ensuring quality in a fast-growing enterprise like online learning is like upgrading the engine on a jetliner 
while it is in flight," says Mickey Revenaugh, Vice President for State Relations at Connections Academy. 
"It's an enormous challenge — but one that virtual program managers must embrace wholeheartedly. If we 
as online educators don't do all we can voluntarily to ensure that we have every possible quality system in 
place, we can be certain that policy-makers and regulators will attempt to do the job for us." 
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Appendix A 



ONLINE LEARNING: 

THE OPPORTUNITIES IT CAN PROVIDE; THE PROBLEMS IT CAN SOLVE 



Below is a list of opportunities that online learning can help provide for your teachers and students and 
problems that online learning can help solve. Please prioritize them by circling the appropriate number. 



Very high 
priority 



1 . Provide methods for differentiating instruction for groups and 
individuals at all grade levels. 

2. Bring more Advanced Placement courses to more students. 

3. Effectively meet the needs of homebound & hospital bound 
students. 

4. Increase the number of electives available to students. 4 

5. Help alternative learning and credit-make up students complete 
necessary coursework. 

6. Engage unengaged students through technology and 4 

independence. 

7. Allow students who are socially uncomfortable to focus on 
academics. 

8. Reduce class sizes by providing more options for students. 

9. Allow students to more easily blend high school and 
post-secondary options. 

10. Allow students to communicate with a wide range of people, 4 

potentially from around the world. 

1 1 . Bring world-class resources and timely information to 
coursework. 

12. Allow students to more easily blend school with duties at home 
or work. 

13. Provide individualized instruction and unique learning 
opportunities. 

14. Increase teachers' familiarity with instructional technologies 
and how to use them in the classroom. 

1 5. Help students use technology as an essential tool for learning. 

1 6. Allow students who don't like school to succeed academically. 4 

1 7. Fill curriculum gaps for any grade or ability level. 

18. Help to fill in areas where available teaching staff is minimal or 4 

requires additional instructional support. 

19. Provide a cost-effective way of providing low-enrollment 
courses. 

20. Allow school-day and calendar flexibility for students. 4 

21 . Allow school-day and calendar flexibility for teachers. 

22. Concretely demonstrate how technology enhances education 4 

to develop support for technology initiatives. 

23. Archive classroom experiences for students, parents or 
accountability. 

24. Modernize curriculum and save money by replacing some or 
all textbook purchases. 

25. Other 4 

26. Other 4 

27. Other 4 



High 

priority 


Low 

priority 


Very low 
priority 


3 


2 


1 


3 


2 


1 


3 


2 


1 


3 


2 


1 


3 


2 
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Appendix B: Resources 

The International Association for K-12 Online Learning (iNACOL) released two important volumes to 
aid online programs in establishing guidelines for online teaching and course quality. The National 
Standards for Quality Online Teaching is a publication that provides states, districts, online programs, 
and other organizations with a set of quality guidelines for online teaching. National Standards of 
Quality for Online Courses provides standards selected on the basis of a research review and survey 
of online course quality criteria. These quality standards were evaluated and assembled into an 
easy to use document for evaluating online courses with common benchmarks. Both documents 
can be downloaded at the iNACOL web site, http://www.iNACOL.org/nationalstandards/index. 
php#teaching. 

Colorado Online Learning posts a number of helpful resources under their Quality Assurance 
Program (QAP), including the full text of the QAP document, http://www.col.k12.co.us/aboutus/ 
qualityassurance.html. 

The VHS Global Consortium program evaluations are available through its website at http://www. 
govhs.org/Pages/WhyVHS-Home. 

National Education Technology Plan, US Dept of Education, http://www.ed.gov/about/offices/list/os/ 
technology/plan/2004/site/edlite-default.htrml, retrieved January 13, 2009. 

Evaluating Online Programs, US Dept of Education, http://www.ed.gov/admins/lead/academic/ 
evalonline/report_pg29.html, retrieved January 14, 2009. 

Essential Principles of High-Quality Online Teaching: Guidelines for Evaluating K-12 Online Teachers, 
Southern Regional Education Board (SREB), http://www.sreb.org/programs/edtech/pubs/PDF/ 
Essential_Principles.pdf, retrieved January 13, 2009. 

A National Primer for K-12 Online Learning , iNACOL, http://www.iNACOL.org/resources/docs/ 
national_report.pdf, retrieved January 12, 2009. 

National Standards of Quality for Online Courses, iNACOL, http://www.iNACOL.org/resources/ 
nationalstandards/, retrieved January 13, 2009. 

Keeping Pace with K-12 Online Learning: A Review of State-Level Policy and Practice, 2008, http:// 
www.kpk12.com. 

Trujillo Commission on Online Education, Donnell-Kay Foundation, http://www.iNACOL.org/ 
resources/docs/TrujilloCommissionOnlineEducationFinalReport-2-1 5-2007.pdf, retrieved January 15, 
2009. 
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